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1.  INTRODUCTION 

A  distributed  computer  program  is  often  a  collection  of  asynchronous, 
sequential  processes  that  occasionally  synchronize  or  communicate.  One 
trend  in  modeling  such  programs  is  to  develop  techniques  powerful  enough 
to  model  arbitrarily  large  programs.  Queueing  network  models  [5]  and 
performance  Petri  nets  [6]  are  principal  techniques. 

A  complementary  approach  is  to  model  small,  elementary  programs  in 
exchange  for  the  ability  to  perform  the  analysis  exactly  Analytic  formulae 
describing  the  exact  run-time  behavior  may  give  fundamental  insights  into 
how  synchronization  affects  distributed  program  performance.  Such  is  the 
approach  of  this  paper.  We  start  by  explaining  the  elementary  type  of 
program  studied  here. 

2  PROGRAM  MODELED 

Consider  a  program  consisting  of  two  processes,  each  of  which  is  executed 
by  an  asynchronous,  dedicated  processor.  Both  processes  share  a  single 
resource,  in  a  manner  to  be  described  shortly.  Each  process  repeats  forever 
the  following  sequence  of  two  local  states: 

(1)  perform  operations  not  requiring  the  resource  (state  0); 

(2)  perform  operations  requiring  the  resource  (state  1).' 


The  i late  name  denole*  the  number  of  resource*  the  process  is  using. 


The  length  of  time  the  process  spends  in  each  state  is  a  known,  positive, 
real  quantity.  The  sum  of  the  time  process  1  spends  in  states  0  and  1  is 
denoted  c^.1  We  define  C2  for  process  2  similarly.  The  time  process  2  uses 
the  resource,  i.e.,  the  time  of  state  1,  is  denoted  by  I  e  (0,  c2).  Variables  c1t 
c2,  and  I  are  real  quantities. 

One  way  in  which  resources  could  be  shared  is  mutually  exclusively: 
neither  process  enters  state  1  and  uses  the  resource,  when  the  other 
process  is  in  state  1.  This  case  is  analyzed  in  [1],  and  requires  a  numerical 
algorithm  to  obtain  the  solution.  The  algorithm  permits  only  an  empirical 
understanding  of  the  solution. 

In  this  paper,  we  analyze  the  more  restrictive  case  of  unilaterally- 
exclusive  resource  sharing  to  obtain  a  closed-form  solution.1  The  closed  form 
exposes  the  functional  dependence  of  behavior  on  parameters  c1t  c2,  and  I. 
In  unilateral  resource  sharing,  only  one  of  the  processes  obeys  the  protocol 
of  not  entering  state  1  when  the  other  process  is  already  in  state  1. 
However,  unlike  mutually-exclusive  sharing,  the  other  process  ignores  the 
protocol  and  uses  the  resource  whenever  it  wishes.  Let  process  1  obey  the 
protocol,  and  process  2  use  the  resource  at  will. 

Two  unilaterally-exclusive  resource  users  together  produce  mutually- 
exclusive  resource  use.  Thus,  the  analysis  of  unilaterally-exclusive  resource 
use  is  a  fundamental  problem. 

1  Quantity  c,  may  also  be  thought  of  as  the  time  required  to  execute  one  cycle  of  process  1  excluding 
the  time  spent  waiting  Tor  the  resource. 

1  The  solution  obtained  is  in  closed  form  to  the  extent  that  the  expression  contains  a  parameter  which 
is  the  solution  of  a  linear  congruence,  and  closed-form  solutions  to  congruences  are  not  known. 
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3.  ANALYSIS  PROBLEMS 

Each  time  process  1  completes  execution  of  state  0,  it  may  need  to  wait 
before  it  can  use  the  resource,  and  thus  enter  state  1.  The  overall  problem 
in  this  paper  is  to  determine  the  functional  dependence  of  the  waiting  time 
on  the  length  of  time  process  2  uses  the  resource  (i.e.,  I).  This  function  is 
denoted  w(l);  its  domain  and  range  are  the  real  numbers.  We  shall  show  that 
the  domain  of  w(l),  which  is  [0,  c2),  may  be  partitioned  into  a  set  of  intervals. 
Function  w  is  constant  within  each  interval,  but  discontinuous  at  interval 
boundaries. 

The  global  state  of  a  program  consists  of  the  local  state  of  all  processes. 
We  describe  the  execution  behavior  of  a  program  by  enumerating  the 
sequence  of 

(1)  global-state  transitions  occurring  during  execution,  and 

(2)  occupancy  times,  or  time  spent  in  each  global  state. 

A  program  is  in  steady  state  when  it  repeats  a  finite  sequence  of  global 
states  and  occupancy  times  forever.  The  time  required  for  each  repetition  of 
the  sequence  is  called  the  steady-state  period. 

In  this  paper,  we  find  for  every  program  fitting  our  model 

(1)  a  proof  that  every  program  reaches  a  steady  state; 

(2)  the  steady-state  period; 

(3)  the  necessary  and  sufficient  conditions  for  a  process  to  wait  for  a 
resource,  given  rational  cycle  times  c1  and  c2;  and 


» 
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(4)  an  expression  for  w(l),  the  time  a  process  waits  each  time  it  uses  the 
resource,  based  on  four  conjectures  about  the  form  of  w. 

Our  chief  analytic  difficulty  is  that  we  only  know  the  global  state  at  the 
moment  both  processes  start  execution.  The  asynchronous  nature  of  the 
processes  precludes  a  priori  knowledge  of  the  global  state  at  any  later  time. 
Therefore,  we  cannot  apply  results  from  scheduling  theory  [2],  critical  path 
analysis  [2],  the  performance  of  flexible  manufacturing  systems  [3],  and 
related  fields  that  analyze  the  time  at  which  a  deterministic  sequence  of 
state  transitions  occurs. 

4.  STEADY-STATE  EQUATION 

In  this  section,  we  analyze  the  steady-state  program  behavior.  The  two 
processes  of  a  distributed  program  may  be  started  at  two  different  times. 
These  two  times  form  the  initial  condition  of  the  program.  First,  we  shall 
discuss  why  any  program  reaches  steady  state  regardless  of  the  initial 
condition.  Then  we  derive  the  condition  under  which  one  process  waits  for 
the  resource  in  steady  state. 

When  a  program  starts  execution  from  any  initial  condition,  one  of  two 
mutually-exclusive  cases  occurs:4 

CASE  1.  Process  1  never  waits  for  the  resource. 

CASE  2.  Process  1  waits  at  least  once  for  the  resource. 


Note  that  process  2  never  wait*  by  definition  of  unilateral  sharing. 
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Consider  case  1  first.  Because  no  process  waits,  process  1  repeats  its  state 
transitions  every  c1  time  units,  and  process  2  repeats  every  c2  units. 
Therefore,  the  moment  both  processes  have  started,  the  program  is  in  steady 
state.  Further,  the  steady-state  period  is  c.,c2  time  units. 

Next,  consider  case  2.  Without  loss  of  generality,  let  time  zero  be  the 
moment  process  2  releases  the  resource  (leaves  state  1),  thereby  permitting 
process  1  to  enter  state  1.  The  processes  progress  in  time  as  shown  in 
Figure  1.  Looking  at  the  progress  of  process  2  in  Figure  1,  the  resource  is 
unavailable  to  process  1  during  time  intervals  [c2  —  I,  c2),  [2c2  —  1, 2c2)  , 

[3c2  —  I,  3c2) .  Meanwhile,  process  1  will  start  using  the  resource  at  times 

0,  c1t  2cj  3c1(  ...  until  some  time  that  is  an  integral  multiple  of  c.,  and  lies 
within  the  interval  when  process  2  uses  the  resource.  Therefore,  process  1 
will  wait  for  the  resource  if  and  only  if 


3  i  e  I,  3  j  e  I  3  ic1  e  [jc2  —  I,  jc2)  , 


where  I  denotes  ti.e  positive  integers.  The  smallest  i  and  j  satisfying  (1) 
represent  the  number  of  loops  processes  1  and  2  make,  respectively,  before 
process  1  waits. 

If  (1)  is  satisfiable,  the  system  will  return  at  time  jc2  to  the  state  it  was  in 
at  time  0.  Thus,  its  behavior  will  repeat,  so  that  (1)  represents  the 
steady-state  behavior.  Alternately,  if  (1)  is  not  satisfiable,  a  desirable 
situation  will  occur  in  which  process  1  waits  exactly  once,  after  which  the 


processes  use  the  resource  without  conflict  and  without  waiting.  This  is  a 
steady-state  behavior  because  the  state  at  time  0  is  returned  to  at  time  c^. 
Thus,  we  have: 


Theorem  1.  A  two-process  distributed  program  in  which  process  1 
unilaterally  shares  a  resource  with  process  2  will  always  reach  a  steady 
state.  Let  the  cycle  times  be  c1  and  c2.  If  waiting  occurs,  the  steady-state 
period  will  be  jc2,  where  j  is  the  smallest,  positive  integer  satisfying  (1). 
Otherwise,  neither  process  waits  for  the  resource,  and  the  period  is  c^. 

Expression  (1)  is  equivalent  to  the  condition  that  process  1  will  wait  if 
and  only  if 


3i  e  I,  3i  6  I,  3x  e  (0,1]  a  ic-,  -  jc2  +  x  =  0  (2) 

Equation  (2)  will  either  have  no  solution  or  an  infinite  number  of  solutions 
[4],  However,  the  solution  corresponding  to  the  program  behavior  is  the 
minimum  positive  integers  i  and  j  satisfying  (2),  for  the  following 
reason:  The  first  time  process  1  leaves  state  0  and  the  resource  is  in  use, 
process  1  will  wait.  This  corresponds  to  the  minimum  number  of  loops  each 
process  makes  before  contending  for  the  resource,  and  hence  the  minimum 
positive  integers  i  and  j  satisfying  (2).s 


Larger,  positive  values  of  I  and  j  correspond  to  physically  unrealir-nhle  situations,  in  which  the 
first  n  (n  >  0)  times  process  1  leaves  state  0  and  the  resource  is  in  use,  process  1  illegally  uses  the 
resource  without  waiting. 
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By  definition,  w(l)  is  the  value  of  x  which  satisfies  (2)  when  the 
minimum  values  of  i  and  j  are  substituted. 

Solving  equation  (2)  presents  three  problems: 

(1)  obtaining  the  condition  under  which  a  solution  exists; 

(2)  allowing  only  integer  values  for  the  solution  of  variables  i  and  j, 
and; 

(3)  finding  the  minimal  values  of  i  and  j  satisfying  (2). 

The  first  two  problems  are  solved  below  by  transforming  (2)  to  a 
Diophantine  equation,6  and  applying  known  methods.  However,  the  third 
problem  is  difficult.  To  our  knowledge,  finding  minimum  integer  solutions 
analytically  is  not  addressed  in  the  literature.  Solutions  to  these  problems 
are  presented  next. 

Earlier,  we  defined  c1  and  c2  to  be  real  quantities.  For  the  following 
analysis,  we  shall  assume  that  c1  and  c2  are  rational  quantities.7  Henceforth, 
we  shall  equivalently  assume  that  c.j  and  c2  are  relatively  prime  integers. 

To  demonstrate  the  equivalence,  we  can  multiply  each  term  in  (2)  by  the 
factor  L/G,  where  L  is  the  least  common  denominator  of  c1  and  c2,  and 
G  is  the  greatest  common  divisor  (g.c.d.)  of  c1  and  c2  .  The  resulting 
solution  to  i,  j,  and  x  in  (2)  must  be  multiplied  by  G.  Finally,  I  must  be 
divided  by  L. 


A  Diophantine  equation  has  integer  coefficients  and  unknowns,  as  discussed  in  [4]. 

The  subsequent  results  do  not  apply  to  processes  whose  cycle  time  is  irrational.  Although  this  case  is 
theoretically  interesting,  in  practice  we  measure  programs  using  only  rational  numbers  (e.g.,  all 
measurements  have  a  precision  of  one  microsecond). 


Because  quantities  i,  j,  c-j,  and  c2  are  integers,  x  must  only  have  integers 
as  its  range.  Thus,  equation  (2)  may  be  considered  to  be  a  Diophantine 
equation  [4]. 

Now,  in  the  following  theorem  and  corollary,  we  may  answer  a  question 
of  practical  importance: 

How  can  we  design  a  two-process,  deterministic  program  to  unilaterally 
share  a  resource  without  conflict  in  steady  state? 

We  seek  the  condition  under  which  the  periods  of  resource  use  of  the 
two  processes  are  interleaved  in  such  a  way  that  neither  process  waits  in 
steady  state. 

Theorem  2.  Consider  a  two-process  distributed  program  in  which  process 
1  unilaterally  shares  a  resource  with  process  2,  and  whose  cycle  times  c1 
and  c2  are  rational  quantities.  At  steady  state,  neither  process  will  ever  wait 
for  the  resource  if  and  only  if  the  duration  for  which  process  2  uses  the 
resource  (i.e.,  \)  is  less  than  the  g.c.d.  of  c1  and  c2. 

Proof.  The  definition  of  unilateral  sharing  only  permits  process  1  to  wait. 
Process  1  waits  if  and  only  if  Diophantine  equation  (2)  has  a  solution. 
Theorem  3.3  in  [4]  gives  the  condition  for  solution: 

A  necessary  and  sufficient  condition  for  the  Diophantine  equation 
a1z1  +  a2z2  +  ...  -I-  anzn  =  k  to  have  a  solution  is  that  the  greatest 
common  divisor  (g.c.d.)  of  the  Sj's  divides  k. 


Let  c  denote  the  g.c.d.  of  c1  and  c2.  If  we  consider  for  a  moment  x 
to  be  an  integer  constant  k,  then  by  the  above  theorem  equation  (2)  has  a 
solution  if  and  only  if  c  divides  k.  Now  k,  or  equivalently  x,  assumes 
any  integer  in  the  range  (0. 1],  Thus,  equation  (2)  has  a  solution  if  and  only  if 
l£c.  □ 

Corollary.  If  a  two-process  distributed  program  unilaterally  sharing  a 
resource  has  equal  cycle  lengths  (c1  =  c^).  then  the  steady  state  never 
requires  either  process  to  wait  for  the  resource. 

Proof.  Neither  process  waits  if  and  only  if  equation  (2)  has  no  solution 
A  solution  exists  if  and  only  if  the  g.c.d.  of  c1  and  c2,  which  is  c2.  divides 
some  integer  in  the  range  (0,1].  Because  the  time  a  process  spends  in  state 
0  and  1  must  be  positive,  I  <  c2.  Thus,  no  such  integer  exists  □ 

Next,  we  find  all  solutions  to  (2).  Using  the  technique  described  in  [4], 
page  66-68,  the  infinite  number  of  solutions  to  (2)  are 

I  =  c2y  +  tx  (3a) 

j  =  c,y  +  rx  ,  (3b) 

where  y  can  assume  any  integer  value  and  r  and  t  are  any  integers 
satisfying1 

c2r  -  Cjt  =  1  .  (3c) 


A  solution  to  (3c)  may  be  found  by  solving  the  following  linear  congruence  by  known  method* 
CjT  =  1  (mod  c,). 
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The  value  of  x  that  minimizes  i  will  also  be  the  value  minimizing  j 
(refer  to  Figure  1).  Henceforth,  we  need  only  analyze  (3a)  Parameter  y 
may  be  eliminated  by  using  conditions  i  ^  0  and  j  ^  0  Combining  these  with 
(3a)  and  (3b)  yields 


_  (  -tx  -rx  \ 

>  4  (  it-  — ) 

0 

From  (3c).  fhe  first  term  in  the  maximum  is  larger  Using  the  notation  [x J 
to  represent  the  largest  integer  that  is  less  than  or  equal  to  x.  and 
substituting  into  (3a)  yields 


This  is  equivalent  to 


(4) 


i  =  (tx  mod  c2)  (5) 

5  EXPRESSION  FOR  FUNCTION  w(l) 

To  find  the  steady-state  waiting  time  wfl).  we  started  in  equation  (2)  by 
seeking  the  minimum  number  of  loops  each  process  makes  before  resource 
contention,  denoted  i  and  j  Knowing  these.  w(l)  is  the  corresponding 
resource  waiting  time  x  We  transformed  this  problem  into  (5).  which  says 


that  w(l)  i*  the  value  xe(0,l]  minimizing  i  This  minimum  is  investigated 
below  to  develop  an  expression  for  w(l) 

First,  observe  that  w(l)  is  a  step  function  To  see  this,  suppose  the  value 
of  I  is  the  constant  l0.  Then  the  value  w(l0)  is  the  integer  in  the  range  <0. 10] 
minimizing  i  in  (5)  If  I  is  increased  to  l0  +  Al  .  where  Al  is  a  real 
number  in  (0,  c2  -  l0)  .  then  w(l0  +  Al)  may  still  be  equal  to  wfl0)  This 
occurs  if  and  only  if  (tx  mod  c2)  >(tv>KI0)  rood  c2),  VX  e  (l0.  I0  +  Al]  Thus.  \MI) 
is  a  step  function 

The  crucial  problem  is  to  determine  the  critical  points  of  w.  or  those 
values  of  I  at  which  w  is  discontinuous  A  critical  point  is  defined  to  be  an 

integer  c  e  (0.c2]  such  that  (tc  mod  c2)  ^  (tz  mod  c2),  Vz  =  1.2 c  We  shall 

define  0  to  be  a  critical  point,  as  a  convenience  in  our  derivation  Thus,  the 
critical  points  are  the  successive  minima  in  a  graph  of  i  versus  x.  as  we 
scan  the  graph  from  x  =  0  to  x  =  c2. 

Example.  Despite  its  simple  appea;ance,  function  i  =  (tx  mod  c2) 
generates  a  rich  variety  of  functions,  as  illustrated  in  Figure  2  Henceforth, 
we  only  use  the  case  of  ^*29,  c2a=50  (Figure  2b)  for  illustrating  the 
conjectures.  An  example  of  the  function  w  is  illustrated  in  Figure  3. 
corresponding  to  Figure  2b  The  critical  points  occur  at  0.  1.  2.  5,  13.  21.  and 
50 

To  find  the  critical  points,  we  make  four  conjectures  about  the  form  of 
function  (tx  mod  c2)  The  resultant  expression  has  been  successfully 


compared  to  the  numerical  evaluation  of  critical  points  by  testing  all  values 
of  x  e  (0,  c2]  in  (5)  at  numerous  values  of  c1  and  c2 .* 

Conjecture  1 .  In  a  graph  of  I  versus  x.  the  critical  points  lie  on  one 
of  a  sequence  of  S  line  segments,  where  S  is  a  positive  integer  If  S  >  1 
each  line  segment  se  [1,2 . S-  1]  shares  an  end  point  with  line  segment 

S  +  1. 

-  One  end  point  of  segment  1  is  not  shared  with  any  other  segment  It  is 
the  point  x  —  0.  i'mcJ 

-  One  end  point  of  segment  S  is  not  shared  with  any  other  segment  It 
is  the  point  x  =  c2.  »*0 

Example  Consider  the  graph  of  equation  (5).  in  Figure  2b  Its  critical 
points  (x-0,  1,  2,  5,  13.  21.  50)  lie  on  S  *  4  line  segments,  as  illustrated  in 
Figure  4  Points  x  =  0,  i  =  50  and  x  =  2.  i  =  12  are  the  end  points  of  segment 
s=1.  and  points  x  =  2. 1  =  12  and  x  =  5.  i  =  5  are  the  end  points  o f  segment 
s  =  2 

Conjecture  2.  On  any  single  line  segment,  the  difference  between  the 
x-coordinates  of  any  two  consecutive  critical  points  is  the  same 

The  two  conjectures  suggest  describing  the  critical  points  along  a  single 
line  by  three  integers  ms.  n,  and  et  The  x  axis  distance  between  two 
consecutive  critical  points  is  denoted  ma  We  denote  by  n,  ♦  1  the  number  of 

*  Specifically.  wc  tatted  all  integer  cwnhuiiUnm  of  c,  in  the  range  (1,1 00]  and  ail  c,  in  l he  range 
[3.100]  Than  are  tatted  ail  integer  combination*  of  c,  m  the  range  of  [1000,  1 100]  and  all  in  ihe 
range  of  [1)00.  1400]  Finally.  we  tested  about  V)  random  point*  for  which  we  cnnrlriKted  and 
studied  graphs  of  i  versus  s  m  the  process  of  formulating  our  i  overtures  these  random  values 
■rare  m  three  categories  r,.  c,  e  J I  .M).  c,.  t,  *  ( 100.  VW).  and  c,  <  t  •  (  10000. 1  I Orsn ) 


critical  points  along  line  segment  s."  Finally,  the  x  coordinate  of  the 
rightmost  end  point  on  line  segment  s  is  denoted  et. 

Example  Returning  to  Figure  4,  the  rightmost  end  points  are  e,  =  2, 
e2  =  5,  e3  =  21,  and  e4  ^  50  The  number  of  critical  points  on  each  segment 

is  nt  +  1  *  3,  n2  +  1  =  2,  n3  +  1  *  3.  and  n4+1-2  Finally,  the  distance 

between  successive  critical  points  on  the  first  line  segment  (x*0,1,2)  is 
m,  =  1 ,  and  on  the  remaining  line  segments  m2  =  1,  m3  =  8,  and  m4  -  29 
If  we  knew  how  to  calculate  mt.  nv  and  es  for  se  [1.S],  then  we  could 
find  all  critical  points  and  hence  function  wfl)  The  remaining  two  conjectures 
describe  how  to  overlay  a  graph  of  (4)  with  a  grid  to  calculate 
mt.  nv  and  es 

Conjecture  3.  It  is  possible  to  draw  two  sets  of  parallel  lines  on  a  graph 
of  i  versus  x  such  that  the  intersection  points  are  exactly  all  points  in 

the  graph  of  equation  (5)  One  set  of  lines  has  positive  slope  and  the 

second  set  negative  slope  Further  the  parallel  lines  of  each  set  are  drawn 
with  equal  spacing  between  them 

Example  The  parallel  lines  of  Conjecture  3  form  a  grid  as  illustrated  for 
the  case  of  ct  =  29  c2  =  50  in  Figure  5  The  parallel  lines  in  each  of  the  two 
sets  are  assigned  consecutive  integer  numbers,  denoted  u  and  v.  as 
illustrated  in  Figure  5 

Conjecture  4.  If  we  know  the  critical  points  along  segment  s  (for  1  <  s 
<  S).  we  can  find  the  slope  of  line  segment  s  +  1  by  drawing  the  grid  of 

*  Thf  critical  point  ciptcuKxi  given  at  the  end  of  thif  lection  u  nmplified  b>  choonng  n,  *  I .  rather 
than  to  represent  the  number  of  critical  point*  along  «egmenl  « 
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Conjecture  3  in  the  following  manner:  The  lines  of  negative  slope  are  drawn 
with  the  line  u-1  containing  line  segment  s.  The  lines  of  positive  slope  are 

drawn  with  the  line  v-0  passing  through  the  origin  and  through  the 

rightmost  end  point  of  line  segment  s.  The  slope  of  line  segment  s  +  1  is 
determined  by  two  points: 

(a)  One  point  is  the  intersection  of  lines  v*0  and  u=1 

(b)  The  other  point  is  the  intersection  of  v=1  and  the  leftmost 
intersection  of  v*1  with  any  line  u. 

Example  Figure  5  illustrates  the  application  of  Conjectures  3  and  4  to 
find  the  critical  points  along  segment  $+1=2  The  lines  of  negative  slope 
are  drawn  parallel  to  line  segment  s  *  1  The  lines  of  positive  slope  are 
drawn  with  the  line  v  =  0  passing  through  the  origin  and  the  rightmost  end 
point  of  segment  s=  1.  which  is  x  =  2,  i  =  12  This  point  is  also  one  end 
point  of  line  segment  2.  by  A  above  Further,  by  B  above,  another  point 
on  segment  2  lies  on  the  line  v  =  1  In  particular,  it  lies  at  the  intersection  of 
v  =  1  with  u  =  2.  because  there  is  no  line  with  a  smaller  value  of  u  that 
intersects  line  v  =  1  Thus,  points  x  =  2,  i  =  12  and  x  =  5.  i  =  5  determine  the 

slope  of  line  segment  2  these  points  are  indicated  by  the  two  shaded  black 

boxes  in  Figure  5 

Using  Conjectures  1  and  4  we  can  obtain  the  number  of  critical  points 
along  line  segment  s  (i  e  .  n,  +1)  and  hence  the  rightmost  end  point  e,  in 
the  following  manner  Conjectures  4  (a)  and  (b)  describe  how  to  find  the  two 
leftmost  points  on  line  segment  s.  and  hence  its  slope  We  can  draw  a  line 
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X  through  the  points  specified  in  Conjectures  4  (a)  and  (b);  thus  it  contains 
line  segment  s.  By  elementary  geometry,  we  calculate  where  X  intersects 
the  x-axis.  From  Conjecture  2,  we  know  that  the  distance  between 
consecutive  critical  points  along  line  s  is  the  constant  ms.  Thus,  we  can 
calculate  the  maximum  number  of  points  which  will  fit  on  the  portion  of  the 
line  through  segment  s  above  the  x-axis,  which  is  ns  +  1. 

Example.  In  Figure  5,  we  can  calculate  n2  -f  1  as  follows.  Line  segment  2 
has  slope  -7/3.  Thus,  a  line  X  containing  segment  2  has  equation 
i  =  50/3  -  7/3x  (see  Figure  6).  Line  X  intersects  the  x-axis  at  x  *  7  1/7  The 
two  critical  points  known  from  Conjectures  4  (a)  and  (b)  occur  at  x  =  2  and 
x  =  5;  therefore  by  conjecture  2.  m2  =  3  If  there  were  more  critical  points  on 

segment  2,  they  would  have  to  occur  at  x  =  8, 11,  14 .  However,  because  X 

intersects  the  x-axis  at  x  «  7  1/4,  no  critical  points  can  have  a  value  larger 
than  7  1/4 

To  summarize,  the  general  procedure  for  calculating  the  critical  points 
presented  in  this  section  is 

(1)  Set  s=  1  Calculate  the  end  points  of  line  segment  1 

(2)  Increment  s 

(3)  Use  Conjecture  4  to  calculate  two  points  on  line  segment  s  +  1 

Set  m,  4  1  to  the  difference  between  the  x  coordinates  of  the  two 
points  (Figure  5) 

(4)  Based  on  the  previous  step,  we  calculate  the  rightmost  end  point  of  line 
segment  s  +  1  (I  e  ,  eg+1)  and  the  number  of  critical  points  on  line 


segment  s  +  1  (i.e.,  ns  +1)  (Figure  6) 

(5)  Steps  2  through  4  are  repeated  until  we  generate  c2  as  the  rightmost 
end  point  of  some  line  segment  (i.e.,  es  =  c2).  The  number  of  segments, 
S,  is  set  to  the  current  value  of  s. 

This  procedure  yields  the  expression  shown  below,  ihe  derivation  is 
discussed  in  the  Appendix. 

Critical-point  expression.  Consider  a  two-process  distributed  program  in 


which 

process  1  unilaterally  shares  a 

resource 

with  process 

2.  Let 

Kx)  = 

tx  mod  c2,  s  €  [1,  S],k  e  [1,  n8].  At  steady  state, 

the  resource 

waiting 

time  v^l)  will  be 

+ 

T 

CO 

© 

II 

I 

m8k  , 

where: 

m,  = 

1 

rfa*_2)  _  *(es_iX1  +  ns-1)l 

+  ms-1 

if  s  =  1 

(6a) 

e,_1|  ns-i  f(e,_1) 

otherwise 

(6b) 

1  C?  1 

L  c2  -  t  J 

if  s  =  1 

(7a) 

n,  - 

,  ®«-1  ^®8-l)ns-1 

1 

otherwise 

(7b) 

Le8_1Ke8_2)  -  e8_2l(e8_1)-  m9  ns_1  Ke,.,)  J 

8 

«.  -  Zn* 

mk 

(8) 

S  satisfies  es  -  c2 


(9) 
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f(e0)  is  defined  to  be  c2 

and  s  and  k  are  integers  chosen  to  satisfy 

^  f[es_1  4-  msk,  es)  jf  k=  ns 

€  Ueg.!  4-  msk,  es_i  4-  ms(k+1))  if  k<ns 

6.  IMPLICATIONS  OF  ANALYSIS 

We  obtained  two  results.  First,  we  showed  that  all  programs  fitting  our  model 
reach  a  cyclic  steady  state.  Next,  we  derived  an  expression  for  w(l),  the 
waiting  time  of  process  1  as  a  function  of  how  long  process  2  uses  the 
resource.  As  illustrated  in  Figure  3,  qualitatively  w(l)  is  a  step  function,  with 
the  width  of  each  step  increasing  with  I.  These  results  have  the  following 
implications: 

(1)  The  waiting  time  w(l)  is  monotonically  increasing.  Therefore,  a  program 
should  have  short  resource-use  durations  to  minimize  the  waiting  time. 
We  proved  that  the  limiting  case  of  zero  waiting  time  occurs  when  the 
resource-use  duration  is  less  than  the  greatest  common  divisor  of  the 
cycle  lengths  of  the  two  processes. 

(2)  We  define  a  program  as  being  unstable  if  a  small  change  in  the 

resource  use  duration  causes  the  waiting  time  to  cross  a  discontinuity, 

thereby  sharply  increasing  or  decreasing.  (For  example,  a  program 
operating  at  1  =  3  in  Figure  3  is  unstable.)  The  space  between 

discontinuities  in  w(l)  grows  as  I  grows.  This  implies  that  a  program  is 


stabler  if  its  resource-use  duration  is  long.  With  increasing  I,  the 
likelihood  that  the  program  is  near  a  discontinuity  decreases. 

(3)  No  waiting  occurs  during  the  transient  period,  before  the  program 
enters  steady  state."  If  the  program  only  needs  to  run  for  a  finite  period 
of  time,  we  can  try  to  make  the  transient  as  long  as  possible,  by 
decreasing  /,  to  avoid  contention.  The  transient  duration  increases  to  a 
maximum  of  c^  as  I  shrinks  to  the  g.c.d.  of  c1  and  c2.,J 

This  work  has  a  broader  implication  for  modeling  deterministic  distributed 
programs  in  general:  the  program  displays  potentially  unpredictable  behavior 
for  small  changes  in  parameters.  This  contrasts  with  continuous  systems, 
such  as  electrical  circuits  modeled  by  a  differential  equation.  A  nice  properly 
of  continuous  systems  is  that  whenever  a  parameter  of  the  system  is 
changed  slightly,  the  output  behavior  changes  in  a  smooth,  predictable 
manner. 

For  example,  consider  the  following  enumeration  of  critical  points  using 
the  expression  developed  for  w(l): 


Processes  enter  steady  state  the  first  time  they  contend  for  the  resource;  therefore  no  resource 
contention  occurs  during  the  transient. 

Graphs  of  equation  (5)  of  i  versus  x  in  Figure  2  illustrate  that  the  number  of  loops,  i,  assumes 
values  corresponding  to  the  successive  minimums  of  f(x)  -  lx  mod  cr  Thus  a  small  value  of  I 
assures  that  x  e  (0,1]  will  correspond  to  a  large  value  of  i.  The  maximum  transient  period  is,  by  (5), 
Cj  loops  of  process  1.  Because  process  I  makes  one  loop  in  time  c,,  the  maximum  transient  duration 
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C1  c2 


critical  values  of  I  at  which  steady-state  cycle  changes 


77 

90 

1 

13 

77 

91 

7 

14 

77 

92 

1 

3 

5 

7  9  11  13  15 

77 

93 

1 

2 

3 
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If  a  programmer  is  running  the  program  with  c^  =  77  and  c2  =  91,  then, 
depending  on  whether  the  value  of  I  chosen  lies  in  the  region 
I  e  [0,7),  [7,14),  or  [14,90),  one  of  three  steady-state  behaviors  will  appear. 
But  if  c2  is  increased  by  just  1,  suddenly  nine  regions  of  I  will  emerge, 
corresponding  to  nine  alternate  steady-state  behaviors.  The  region  [0,7)  is 
now  split  into  four  regions,  and  region  [7,14)  is  divided  into  three  regions.  If 
c2  is  again  increased  by  one,  the  number  of  regions  will  shrink  to  five. 

The  erratic  change  in  number  of  regions  suggests  how  complex  the 
microscopic  behavior  underlying  some  distributed  systems  may  be.  This 
raises  the  question  of  whether  distributed  programs  with  more  processes 
and  resources  are  more  or  less  erratic.  Experimental  results  in  [1]  suggest 
that  aggregation  of  many  microscopic  discontinuities  blends  into  a  system 
which  macroscopically  displays  relatively  few  points  of  discontinuity.  This  is 
encouraging  for  further  analysis. 


.*  *  *  <  v 


7  APPENDIX 


Summarized  below  is  the  erivation  of  expressions  for  the  distance  between 
critical  points  (ms)  and  the  number  of  critical  points  (ns+1)  for  each  line 
segment  s.  The  complete  derivation  appears  in  [1].  First,  we  shall  consider 
line  segment  s  =  1  and  derive  (6a)  and  (7a).  Later,  we  shall  derive  (6b)  and 
(7b),  for  s  >  1,  by  carrying  out  the  five-step  procedure  given  in  Section  5. 

A.1 .  Critical  Points  on  Line  Segment  s  =  1 

In  Conjecture  1,  we  defined  one  end  point  of  line  segment  s  =  1  to  be  the 
point  x  =  0,  i  =  c2.  By  definition  x  =  1,  i  =  t  is  a  critical  point.  Thus,  the  slope 
of  line  1  is  —  (c2  — t)/1,  and  m^  =  1.  Now  we  want  to  know  how  many  critical 
points  x  =  2,  3,  ...,  n1  also  lie  on  this  line. 

One  way  to  view  function  f(x)  =  tx  mod  c2  is  as  a  graph  of  tx  "folded 
over"  on  itself  every  c2  units  (Figure  7).  Horizontal  lines  are  drawn  in  the 
figure  at  integral  multiples  of  c2.  The  values  of  f(x)  may  be  derived 
graphically  using  Figure  7  by  calculating,  for  each  x,  the  distance  from  tx  to 
the  previous  multiple  of  c2.  We  can  show  that 


■  t(x+  1)  . 

L  co  J 


I  — I  +1  if  x  is  a  critical  point 

I  tx  I 

L  c2  J  otherwise 


(A1) 


In  terms  of  Figure  7,  the  point  x  =  2  is  critical,  while  3  is  not,  because 
[  7  3/11  J  =  [  7-2/11  J.  This  gives  us  a  condition  for  finding  the  smallest 


critical  points:  the  points  1,2,...,n  sub  1  are  critical,  where  n1  +  1  is  the 
smallest  integer  for  which  [t(n1  +  1)/c2J  =  [tn1/c2J .  Because  (4)  is 
equivalent  to  (5),  f(x)  =  tx  -  c2  |_tx/c2J  .  Thus,  at  x  =  n1  4-  1 


f(x)  =  t(n1  +  1)  - 


c2 


|t(n2  +  1) 

L  c2 


Substituting  (A1)  yields 


f(x)  =  t(ni  +  1)  -  c^-1)  .  (A2) 

Because  0  <  f(x)<  c2,  substituting  (A2)  for  f(x)  and  solving  for  ^  yields  (7a). 

A.2.  Critical  Points  on  Line  Segments  s  >  1 

Now  we  proceed  to  the  case  of  s>  1.  First,  we  obtain  an  invertible  relation 
between  f(x)=  tx  mod  c2  and  parameters  u  and  v 


f(x)  =  afu  4-  bfv 
x  =  ax  u  +  bx  v  . 


(A3) 


To  determine  parameters  af,  bf,  ax,  and  bx,  we  need  two  points,  other  than 
the  origin,  for  which  the  u  and  v  parameters  are  known.  Lines  u  =  1 
and  v  =  0  always  pass  through  the  critical  point  at  x  = 
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f(es-i)  =  af  ‘1  +  bf  0 
es_ i  =  aw-1  +  bw0 


(A4) 


The  lines  u  =  1  and  v  =  -  ns-1  always  pass  through  the  critical  point  at 
x  =  es_2: 


f(es_2)  =  ar1  -  bfns_, 

es-2  =  ax +  bx0  . 


(A5) 


Solving  (A4)  and  (A5)  for  afl  bf,  ax,  and  bx,  and  then  substituting  into  (A3) 
yields  the  desired  relation 


f(x)  =  f(es_1)u  + 


f(es-i)-f(es_2) 

ns— 1 


(A6a) 


x  =  es,^  +  .  (A6b) 

Using  (A6)  with  the  conjectures,  we  shall  find  expressions  for  ms  and  ns. 
First,  we  obtain  two  functions  which  simplify  the  derivation:  u(v)  and  x(v). 

Suppose  we  wish  to  find  the  critical  points  along  line  segment  s.  Each 
critical  point  lies  on  lines  v  =  1,2,...,ns,  at  the  point  with  the  minimum  f(x) 
value.  Function  u(v)  is  defined  to  be  the  value  of  u  such  that  the 
intersection  of  v  and  u(v)  is  the  point  with  the  minimum  f(x)  value  along 


23 


line  v.  We  can  find  u(v)  from  the  condition  that  f(x)  ^  0.  We  substitute  (A6a) 
for  f(x),  then  solve  the  inequality  for  u(v).  The  minimum  integer  satisfying  the 
inequality  is 


u(v) 


*es_2)  —  i ) 

ns-l^es-l) 


(A7) 


Function  x(v),  when  given  a  value  v  on  which  a  critical  point  lies,  yields 
the  x  coordinate  of  that  critical  point.  This  function  is  (A6b)  with  function 
u(v)  substituted  for  u: 


x(v)  = 


•f(es_2)~<(es-i)  ' 
ns_1f(es_1) 


+  ms_lV 


Next,  we  obtain  the  expression  for  ms,  the  spacing  between  two  critical 
points  on  the  same  line  segment,  by  subtracting  the  x  coordinates  of  two 
adjacent  critical  points  on  line  segment  s.  Each  line  segment  contains  a 
critical  point  at  its  leftmost  end  point  (x  =  e^).  Further,  the  line  v=  1  must 
pass  through  another  critical  point.  Function  x(v),  evaluated  at  v=  1,  yields 
the  x  coordinate  of  this  point.  Thus,  the  difference  between  two  critical 
points  on  line  s  is 


m8  =  x(1)  -  es_1  . 
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Substituting  (A7)  for  x(v)  and  simplifying  yields 


m 


s 


*c8_2)-  tie,,,)- 


1  +  ms-1V 


Finally,  we  find  an  expression  for  n.  Each  line  segment  containing  critical 
points  has  a  negative  slope  Thus,  if  we  fix  v  and  choose  u  too  large  in 
(A6a),  then  a  negative  value  of  ffx).  as  defined  in  (A6a).  results  Thus  we 
can  find  the  number  of  critical  points  along  line  s  by  finding  the  largest 
integer  ns  for  which  f(x)^  0  Substituting  (A6a)  yields 

ffe,_1)  -  Ke,_2) 

He^Mn,)  +  — ^ — ns  2?  0  <A8> 


We  can  eliminate  u(n9)  by  solving  (A6b)  for  u  Further  the  x  coordinate 
of  the  rightmost  critical  point  along  line  s  must  be  greater  by  nsms  than 
the  leftmost  critical  point  on  line  s,  at  x  =  e5_1  Thus 

n.m.  4-  e.  .  —  m_  ^n, 

u(n,)  =  -i- - - ~ - 

®s-i 

Substituting  the  above  in  (A8)  and  solving  for  ns  yields 


ASIA 
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ns  “ 


*e8-i)[(ms-i  -  + 


i«*s 


-2' 


Substituting  e9_2  tor  m9_1n9_1  -  yields  (7b) 
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I  =  31x  mod  50 


Segment  s  =  1  —  Segment  s  =  2 


FIG.  4.  Illustration  of  conjecture  1.  All  critical  points  lie  on  one  of  four  line  segments. 


Segment  s  =  1 


v 
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FIG.  5.  Illustration  of  grid  of  conjectures  3  and  4  overlaying  graph  of  Figure  2b.  Grid  is 
oriented  to  obtain  the  critical  points  x  =  2  and  x  =  5  lying  on  line  segment  s  +  1 
=  2.  The  two  black  circles  represent  the  two  points  specified  in  Conjectures  4  A 
and  B  that  determine  the  slope  of  segment  s  +  1. 
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